Methods, devices and computer program products for capture and display of visually encoded data and an image

ABSTRACT

Visually encoded data is captured along with one or more visual elements, the encoded data is decoded and one composite view is displayed that incorporates information related to the decoding data, such as a decode confirmation or decoded data, and the visual elements. The recognition and decoding process may additionally provide for transforming the captured visual elements to accommodate the allocated display area of small sized display, such as the display associated with a handheld image capturing device. The invention is thus suited for capturing encoded data and branding elements, such as company or product logos, and displaying the branding elements in unison with data related to the decoded information.

FIELD OF THE INVENTION

This invention relates to visually encoded data, and more particularly,relates to methods, devices and computer program products that providefor capturing visually encoded data and an associated image element anddisplaying the decoded data and the image element on a handheld digitaldevice.

BACKGROUND OF THE INVENTION

An increasing amount of handheld digital devices, such as, cellulartelephones, personal data assistants (PDAs) and the like are currentlyavailable with image capture devices, i.e., cameras. The users of suchdevices benefit from being able to capture an image. i.e., take aphotograph, at any point in time or any occasion. Since these devicesare typically always in the possession of the user, the opportunity forthe user to capture an image is omnipresent. Along with providing theever-present opportunity to capture the image, these devicesadditionally provide sufficient digital memory to store multiple imagescentral processing unit (CPU) capabilities that provide for processingand analyzing the images and, in many instances, wireless communicationmeans that allow the user to wirelessly communicate the image to otherdigital device users.

However, handheld devices, such as cellular telephones, PDAs and thelike that are equipped with image capture capabilities may not belimited to conventional photo capture. In addition, such devices mayinclude functionality to capture and decode visually encoded data orsymbologies. Visually encoded data typically takes the form of linearbarcodes, 2-dimensional barcodes, steganographically coded data and thelike. Examples of conventional 2-dimensional barcodes include QuickResponse (QR) code and DataMatrix. These types of 2-dimensionalsymbologies allow a large amount of data to be visually encoded in arelatively small area.

For example, in one hypothetical situation, a user of an image captureequipped handheld device may be interested in purchasing a largeappliance or automobile. At the point of sale, typically a retail outletor automotive dealer, the product may be on display along with avisually encoded tag. The visually encoded tag would allow the handhelddevice user to capture, decode and store information related to theproduct, such as product specification data, pricing information and thelike. As is the case in this hypothetical situation many of the visuallyencoded tags that a handheld device will encounter are related to acommercial product or a commercial service that is being offered. Inthis instance, the encoded data is generally associated with acommercial entity. For example, the large appliance may be offered bythe Sony Corporation and the automobile may be offered by Ford MotorCompany.

Taking into account that visually encoded tags are typically providedfor the user's benefit by a commercial entity it would be advantageousto the tag provider, as well as, adding to the overall experience of thehandheld device user to provide visual images, typically brand relatedvisual images, in conjunction with the confirmation view or otherdisplays of the decoded information. One possible solution would be topreload, in the device's memory, the graphics (company logos, fonts,colors and the like) of tag participating companies, i.e., companiesusing visual codes in their advertising or marketing campaigns. Duringdecoding of the encoded data, the decode application would access thememory to choose related graphics. For example, a tag determined to havedata related to the Nokia Corporation would access and display a blueNokia logo in conjunction with the display of the decoded confirmationinformation or other decoded tag data. However, this method is less thanideal because all of the participating companies and brands need toknown beforehand, and if new companies or brands deploy visually encodedtags the digital device would be unable to display matching graphicalelements, since none would be stored in the device's memory.

Another potential solution would be to provide for downloading ofgraphics during the decoding process. In some instances, visuallyencoded data will include commands and instructions for connecting witha remote network to access additional information, for example,initiating a browser and connecting with a company website. In thisexample, it would be possible to connect with the outside networkresource and download the company or brand graphics from the externalstorage site. However, this potential solution would slow down theoverall user experience and the user may incur network connectioncharges associated with the download process.

Additionally, often the recipient of visually encoded data will have adesire to store the decoded information. For example, if the encodeddata includes a coupon or a hyperlink or the link, the user may wish tostore the decoded information for future use or reference. Typically, ifthe user stores voluminous amounts of such data, they may experiencedifficulty in trying to locate the data in memory. This may beespecially evident if the user can not recall the title of the storeddecoded data or the search functions of the application are limited tobrowsing.

Therefore, a need exists to develop a method, system and computerprogram product that will allow for graphical representations, such ascompany or brand information, to be displayed in conjunction withdecoded confirmation information or other decoded data from visuallyencoded tags. The desired application will take into account therelative small size display of handheld held devices, such that theapplication will properly format the data to accommodate the display ofthe graphical representations in conjunction with the decodeconfirmation or other decoded data. The desired method, system andcomputer program will provide for efficient, user-friendly, presentationof the graphical representations on the display, in that, the methodwill not rely on downloading of the graphical representations from anassociated network service. In addition, the desired methods, systemsand computer programs will not be limited to graphical representationsstored in digital device memory but, rather, will provide for graphicalrepresentations that are uniquely associated with the visually encodedtag that is being read and decoded. Additionally, a need exists todevelop a more efficient and user-friendly means for locating, withindevice memory, stored decoded data.

BRIEF SUMMARY OF THE INVENTION

The present invention provides methods, devices and computer programproducts that provide for visually encoded data to be captured alongwith one or more visual elements, for the captured visually encoded datato be decoded and for information related to the decoding data and thevisual elements to be displayed in unison. The application that performsdecoding of the visually encoded data may additionally provide fortransforming the captured visual elements to accommodate the allocateddisplay area of small sized display, such as the display associated witha handheld image capturing device. Thus, the present invention providesfor an efficient and user-friendly method of displaying visual elementsin unison with decoded or decode confirmation information. In addition,the present invention provides for a one-to-one relationship between theencoded data and the visual elements, such that, each tag containingencoded data can have individual and unique visual elements connectedwith and displayable in conjunction with the encoded data.

In one embodiment of the invention a method is defined for capturing,decoding and displaying information encoded and visually represented ona visual encoded data tag. The method includes the step of directing adigital device, which includes an image capturing unit, at a visual tagthat includes a visually encoded data portion and a visual elementportion. The digital device will typically be a handheld digital device,such as a cellular telephone, PDA, image or video camera or the likethat is equipped with an image capturing unit (i.e., a camera unit). Thevisual tag will typically include a first predefined portion thatincludes visually encoded data and a second predefined portion thatincludes one or more visual elements. The visually encoded data may takethe form of a barcode and, in particular, 2-dimensional barcodesymbologies, such as PDF417, Data Matrix, MaxiCode and QR Code or thelike. The one or more visual elements may include brand elements, suchas company, product or service logos or other elements connected to acompany, product or service.

In alternate embodiments, the visual tag may include one integral entitythat includes both a visually encoded portion and a visual elementportion. For example, the geometry of the entity may define the visualelement portion and encoded data is enclosed with the boundaries of thegeometry. In these alternate embodiments, symbologies such as DataGlyphs may be used to encode the data within the geometric shape.Decoding of the one integral entity results in both decoded data and avisual element; i.e., the geometry of the entity without the encodeddata and, possibly, shaded or otherwise filled-in.

The method additionally includes the steps of reading, at the digitaldevice, the visually encoded data and capturing, at the digital device,the one or more visual elements. Reading the visually encoded data andcapturing the visual elements will entail recognition of the predefinedareas of the tag that include the visually encoded data and the visualelements. The method will also include the steps of decoding thevisually encoded data and displaying, on a display associated with thedigital device, a composite view that includes at least a portion of thevisual element portion and information related to the decoded data.Typically, the information related to the decoded data will includedecode confirmation or actual data decoded. This method provides forinformation related to the decoded information to be presented in asingle screenshot along side the visual elements. In the instance inwhich the visual elements are branding elements, this method providesfor the branding elements to be displayed on the digital device inconjunction with decode information.

The method may additional entail the step of formatting the capturedvisual element portion prior to display. Formatting of the visualelement portion provides for such exemplary procedures as resizing thevisual element portion, rotating or realigning the visual elementportion, adjusting visual element characteristics or the like.Formatting of the visual element portion is typically required as ameans of insuring that the visual elements can be properly displayed inthe area allocated for display of such elements.

Additionally, the method may involve the step of initiating networkcommunication as a result of the decoding of the visually encoded data.Examples, of such network communication include, but are not limited to,Short Message Service (SMS) communication, Multimedia Message Service(MMS) communication, electronic mail communication, cellular networkcommunication, and Internet communication. In addition, the method mayinclude storing the decoded data and accompanying visual elements indevice associated memory, otherwise referred to as bookmarking. Inembodiments that include a bookmarking step, the memory may be accessedto retrieve the stored decoded data and provide for the unified displayof the information related to the stored decoded data and the visualelement portion.

The invention is further defined by a visually encoded data tag thatincorporates a first predefined area that encloses visually encoded dataand a second predefined area adjacent to the first predefined area thatencloses one or more visual elements. The visually encoded data may takethe form of a barcode and, in particular, 2-dimensional barcodesymbologies, such as PDF417, Data Matrix, MaxiCode and QR Code or thelike. The one or more visual elements may include brand elements, suchas company, product or service logos or other elements connected to acompany, product or service. The predefined areas of the visual tag maybe defined such that that an associated recognition application iscapable of determining that the data in the first predefined area issubject to decoding and that at least a portion of the one or morevisual elements are subject to user interface display. The visuallyencoded tag may, but is not required to, incorporate a third predefinedarea that is located on the tag adjacent to either or both of the firstand second predefined areas. The third predefined area may include tagidentifying or tag instructions data, i.e., visual data that advises theuser of the digital device to capture the code or informs the user as tothe nature of the code contents. In this instance, the informationincluded in the third predefined area is typically ignored by theassociated recognition application.

In an alternate embodiment of the visually encoded tag, the visualelements may be encoded to include user-interface display parameters.For example, the visual elements may take the form of a series ofgeometric shapes with each shape, position of a shape, color of a shape,shading of a shape or the like, indicating a specific user-interfacedisplay parameter. Examples of user-interface display parameters includefont size, font type, font color, background color, dialog template andthe like, which would, for example, render the user interface to followbrand or product guidelines or follow guidelines of a company's visualimage.

The invention is also embodied in a computer program product forcapturing, decoding and preparing for display visually encoded data andaccompanying visual elements. The computer program is typically executedby a handheld digital device having image capture capabilities, such asa cellular camera telephone or the like. The product includes a computerreadable storage medium having computer-readable program instructionsembodied in the medium. The computer-readable program instructionsincludes first instructions for determining a first predefined area of avisually encoded tag that encloses visually encoded data and a secondpredefined area of the visually encoded tag that encloses one or morevisual elements. In this regard the program instructions recognize andidentify specific regions of the tag; the first predefined regionincludes the visually encoded data and the second predefined regionincludes the visual elements. The program instructions also includesecond instructions for decoding the visually encoded data. The decodingprocess will be associated with the type data encoded that is used. Foeexample, if the data is encoded using a conventional 2-dimensionalbarcode scheme, such as QR Code or Data Matrix, then the secondinstructions will incorporate the requisite related decode algorithm.

The computer-readable program instructions will also include thirdinstructions for extracting at least a portion of the one or more visualelements and applying the extracted portion to a first predefined areaof a user-interface template. One or more user-interface templates aretypically stored in memory associated with the digital device and formthe basis forms the basis for a display view, otherwise referred toherein as a screenshot. The third instructions may additionally providefor applying decoded data or confirmation of decoding to a secondpredefined area of the user-interface template.

Additionally, the third instructions of the computer-readable programinstructions may provide for the transforming at least a portion of theone or more visual elements prior to applying the portion to the firstpredefined area of the user-interface template. Transforming the visualelements provides for the elements to be modified, as necessary, toaccommodate display size and other relevant display features. Forexample, transformation may include, resizing the visual elements,rotating or otherwise reconfiguring the alignment of the visualelements, cropping the visual elements, synthesizing the visualelements, adjusting display parameters associated with the visualelements and the like.

The computer-readable program may additionally include fourthinstructions for displaying a composite view or screenshot, typically onthe digital device display, that uses the user-interface template as anoutline and includes the extracted portion of the one or more visualelements in the first predefined area and, typically, informationrelated to the decoded data, such as decode confirmation or actualdecoded data.

Alternatively, the invention is defined by an alternate computer programproduct for capturing, decoding and preparing for display visuallyencoded data and associated visual elements. The product includes acomputer readable storage medium having computer-readable programinstructions embodied in the medium. The computer-readable programinstructions includes first instructions for determining a firstpredefined area of a visually encoded tag that encloses visually encodedprimary data and a second predefined area of the visually encoded tagthat encloses one or more visual elements that encode user-interfaceparameter data. In this regard, the primary encoded data is typicallyrepresented by a barcode symbology and the visual elements encodeuser-interface parameters. For example, the visual elements may begeometric shape and the parameters are encoded based on the shape of theelement, the location of the element, the shade of the element, thecolor of the element or the like.

The computer-readable program instructions also include secondinstructions for decoding the visually encoded primary data and thevisual elements that encode the user-interface parameters. The decodingprocess will be associated with the type data encoded that is used. Foeexample, if the primary data is encoded using a conventional2-dimensional barcode scheme, such as QR Code or Datamatrix, then thesecond instructions will incorporate the requisite related decodealgorithm.

The computer-readable program instructions will also include thirdinstructions for applying the decoded user-interface parameters to auser-interface template. Applying the decoded user-interface parametersto the user-interface template may entail changing the font type, fontsize, font color or background color or adding other icons or entitiesto the template. The third instructions may additionally provide forinstructions for applying information related to the decoded primarydata, such as a decode confirmation or actual decided data, to adesignated area of the user-interface template.

The computer-readable program may additionally include fourthinstructions for displaying a composite view or screenshot, typically onthe digital device display, that uses the user-interface template as anoutline and includes the applied user interface parameters and,typically, information related to the decoded data, such as decodeconfirmation or actual decoded data.

The invention may also be embodied in a digital device for capturingvisually encoded data and visual elements and displaying decodedinformation in unison with the visual elements. The device willtypically be a handheld digital device such as a cellular telephone,PDA, image or video camera or the like. The device will include an imagecapture device capable of capturing an image from a visually encodedtag, a processing unit in data communication with the image capturedevice that receives image data from the image capture device, a memoryunit in communication with the processing unit that stores one or moreuser-interface templates and a display in communication with theprocessing unit.

Additionally the device will include an application or routine executedby the processing unit. The application is embodied within a computerreadable storage medium that has computer-readable program instructionsembodied within the medium. The computer-readable program instructionsinclude first instructions for determining a first predefined area ofthe image that encloses visually encoded data and a second predefinedarea of the image that encloses one or more visual elements, secondinstructions for decoding the visually encoded data and thirdinstructions for extracting at least a portion of the one or more visualelements and applying the extracted portion to a first predefined areaof one of the stored user-interface templates. The display of the devicewill provides for the display of a composite view or screenshot based onthe stored user-interface template and including the applied portion ofthe one or more visual elements.

Thus, the present invention provides for methods, devices and computerprogram products that provide for visually encoded data to be capturedalong with one or more visual elements, decoding the captured visuallyencoded data and displaying, in one composite view, information relatedto the decoding data and the visual elements. The application thatperforms decoding of the visually encoded data may additionally providefor transforming the captured visual elements to accommodate theallocated display area of small sized display, such as the displayassociated with a handheld image capturing device. The invention is thussuited for capturing encoded data and branding elements, such as companyor product logos and displaying the branding elements in unison withdata related to the decoded information. Thus, the present inventionprovides for an efficient and user-friendly method of displaying visualelements in unison with decoded or decode confirmation information. Inaddition, the present invention provides for a one-to-one relationshipbetween the encoded data and the visual elements, such that, each tagcontaining encoded data can have individual and unique visual elementsconnected with and displayable in conjunction with the encoded data.

BRIEF DESCRIPTION OF THE DRAWINGS

Having thus described the invention in general terms, reference will nowbe made to the accompanying drawings, which are not necessarily drawn toscale.

FIG. 1 is block diagram of a system for encode data capture and display,in accordance with an embodiment of the present invention.

FIG. 2 is an illustration of a visually encoded data tag template, inaccordance with an embodiment of the present invention.

FIG. 3 is an illustration of an example of a visually encoded data tag,in accordance with an embodiment of the present invention.

FIG. 4 is an illustration of a screenshot or view template, inaccordance with an embodiment of the present invention.

FIG. 5 is an illustration of an example of a screenshot or compositeview resulting from the capture and decode of the tag of FIG. 3, inaccordance with an embodiment of the present invention.

FIG. 6 is an illustration of an example of a visually encoded data tagin which the encoded data and the visual elements are embodied in asingle entity, in accordance with an embodiment of the presentinvention.

FIG. 7 is an illustration of an example of a screenshot or compositeview resulting from the capture and decode of the tag of FIG. 6, inaccordance with an embodiment of the present invention.

FIG. 8 is an illustration of an example of a visually encoded tag inwhich the visual elements define coded user-interface displayparameters, in accordance with an embodiment of the present invention.

FIG. 9 is an illustration of an example of a screenshot or compositeview resulting from the capture and decode of the tag of FIG. 8, inaccordance with an embodiment of the present invention.

FIG. 10, is a flow diagram of a method for encoded data capture, decodeand data display, in accordance with an embodiment of the presentinvention.

DETAILED DESCRIPTION OF THE INVENTION

The present invention now will be described more fully hereinafter withreference to the accompanying drawings, in which preferred embodimentsof the invention are shown. This invention may, however, be embodied inmany different forms and should not be construed as limited to theembodiments set forth herein; rather, these embodiments are provided sothat this disclosure will be thorough and complete, and will fullyconvey the scope of the invention to those skilled in the art. Likenumbers refer to like elements throughout.

The present invention provides methods, devices, computer programproducts and systems that provide for visually encoded data to becaptured along with one or more visual elements and for the decodedinformation and the visual elements to be displayed in unison. Theapplication that performs decoding of the visually encoded data mayadditionally provide for transforming the captured visual elements toaccommodate the small sized displays, such as a display associated witha handheld image capturing device.

FIG. 1 provides an overview of the system for data capture and datadisplay, in accordance with an embodiment of the present invention. Adigital device 10 is equipped with an image capture unit 20, such ascamera. The digital device will typically be a handheld digital device,such as a cellular telephone, a personal digital assistant, a digitalcamera, a digital video camera or the like. The image capture unit iscapable of capturing images from visually encoded tag 30. The contentsof the visually encoded tag are paramount to the present invention andwill be explained in more detail below. A processing unit 40 disposedwithin the digital device receives the image data from the image captureunit and executes a recognition and decoding application 50 thatrecognizes predefined areas of the visually encoded tag image, performsrequisite decoding operations and prepares a composite view, alsoreferred to as a screenshot, for display on the digital device display60. The application will access to device memory 70 to retrieve anapplicable user-interface template 80. The user interface template willprovide the outline or baseline for the view that the applicationcompiles. Once the recognition and decoding operation has performeddecoding and presented a decoded data composite view, optional externalapplications 90, such as network communication applications may beinitiated based on commands in the decoded data.

FIGS. 2 and 3 provide illustrations of a visually encoded tag templateand an example of a visually encoded tag, respectively, in accordancewith an embodiment of the present invention. The visually encoded tagtemplate 100 of FIG. 2 defines specific areas for enclosing the visuallyencoded data and for enclosing the visual elements. The specific areasare subsequently recognized by the associated recognition and decodeapplication according to their size and location on the tag. Requisiteprocessing occurs of the information enclosed in a predefined tag areawill occur once the area had been recognized. In the illustratedembodiment of FIG. 2, a first predefined area 110 defines the portion ofthe tag that encloses the visually encoded data. The visually encodeddata may be in the form of any known visual coding symbology. Typically,the encoded data will be in barcode form. Two-dimensional barcodesymbologies may be preferred, as they provide for more information to beencoded in a smaller area. Examples of two-dimensional barcodes includePDF417, Data Matrix, MaxiCode and Quick Response (QR) Code.

The second predefined area 120 of visually encoded tag template 100defines the portion of the tag that encloses one or more visualelements. The term “visual element” is used to define specific visualentities, items or images that are related to the information visuallyencoded in the first predefined area. In one embodiment of the inventionthe visual elements are further defined as branding elements, such asproduct, service or company, logos, symbols, insignia or the like. Aswill be explained in more detail below, the recognition and decodeapplication that is executed by the tag capturing device will recognizethe second predefined area as the area that defines one or more visualelements. The application will then process the visual elementsaccordingly. In one embodiment of the invention, the visual elements maybe optionally formatted, i.e., transformed and presented for display ina composite view or screenshot in unison with information related to thedecoded data, such as a decode conformation or actual decoded data. Inanother embodiment of the invention the nature and characteristics ofthe visual elements, such as shape, location, color, shading or thelike, may define user-interface parameters or settings. In such case,processing of the visual elements will entail setting the user-interfaceparameters according to the characteristics of the visual elements.

The third predefined area 130 of visually encoded tag template 100defines an optional portion of the tag that encloses tag identifyingdata. Tag identifying data is optional visual data that may be placed onthe tag to notify a digital device user that encoded data exists andthat capturing the data may be desirable, i.e., recognizing the visuallyencoded data as a source for action. The data enclosed by the thirdpredefined area is not processed by the recognition and decodeapplication, however, the size and shape of the third predefined areamay be relied upon by the recognition algorithm to locate and identifyeither or both of the first and second predefined areas.

FIG. 3 provides an example of a visually encoded tag 140, in accordancewith an embodiment of the present invention. The first predefined areaencloses visually encoded data 150, in this example, DataMatrix code, aconventional 2-dimensional barcode symbology. The second predefined areaencloses one or more visual elements 160. In the illustrated example thevisual element is a branding element that includes a product logo,“gage” and a company logo, “nokia”. The optional third predefined area170 encloses tag identifying data, in this example, the term “get more”is included to inform a user that potentially-desirable encodedinformation is located on the tag.

FIGS. 4 and 5 provide illustrations of a display screenshot template andan example of display screenshot, in accordance with an embodiment ofthe present invention. As previously noted in the discussion related toFIG. 1, the digital device will capture the image of the visuallyencoded tag and a recognition and decode application will be executed.In one embodiment the application will prepare display screenshots orcomposite views that incorporate at least a portion of the one morevisual elements present in the second predefined area of the visuallyencoded tag. FIG. 4 illustrates an example of screenshot, otherwisereferred to as a view, template 200. The application of the present willselect an appropriate template from one or templates stored in digitaldevice memory. Selection of an applicable template may be based on theshape or size of the visual elements present on the tag or the selectionmay be designated on the tag via an encoded template designator.Alternatively, other modes of template selection may be possible. Thegeneral shape of the templates will be aligned with the size of theviewable area of the display. Thus, in embodiments of the presentinvention in which the tags are captured and decoded on handheld deviceshaving relatively smaller type displays, the associated templates willreflect the shape and size of the display.

The screenshot template of FIG. 4 includes a first region 210 designatedfor visual elements and a second region 220 designated for informationrelated to the decoded data. Typically, in order for the one or morevisual elements of the tag to be included within the first region of thescreenshot template transformation, otherwise referred to asre-formatting, of the graphics will be required. Such transformation ofthe graphics may entail, but is not limited to, rotating orreconfiguring the visual elements, resizing the visual elements,filtering the visual elements to remove distortions, and adjustingvisual display parameters, such as, contrast, tint, color and the like.The second region 220, which is designated for information related tothe decoded data, may include decode confirmation or specific decodedinformation. For example, the second region may include confirmationthat the data has or has not been properly decoded or the second regionmay provide for decoded data, such as, for example, instructions for theuser to initiate a network connection.

FIG. 5 provides an example of a display screenshot 230 resulting fromcapturing of the visually encoded tag 100 of FIG. 3 and using thetemplate 200 of FIG. 4 as the outline. The dotted lines are illustratedto show the corresponding template regions and are not actually depictedin the displayed screenshot. The first region 210 includes visualelement 160. The visual element has been transformed, i.e., re-formattedto fit the size constraints of the first region. The second region 220includes information related to the decoded data 240. In thisillustrated example, the information is decoded data, in the form of ahelp-line, referred to in the screenshot as a hotline, telephone number.The user of the digital device, upon being provided with the informationin the screenshot may choose to activate the network communication;i.e., dial the helpline number or the user may choose to bookmark thedecoded data. Bookmarking of the decoded data will result in thedisplayed screenshot being stored in device memory for future access anduse.

In an alternate embodiment of the present invention, the visuallyencoded data tag will represent the visual elements and the encoded datacollectively as a unified object. For example, FIG. 6 depicts a visuallyencoded data tag 300, in which the visual elements and the visuallyencoded data are represented in a single unified object, in accordancewith an embodiment of the present invention. The visually encoded data310 is embedded in the geometric shape. An example of a coded symbologythat provides for embedding the code in a picture or a shape isDataglyphs, available from PARC Solutions, of Palo Alto, Calif. Thevisual element 320 is the outline of the geometry that encloses thevisually encoded data. In the illustrated embodiment the outline of theshape provides for the formation of a company name “Nokia”.

The visually encoded data tag 300 of FIG. 6 will be captured, decodedand displayed in the following manner. A digital device equipped with animage capture unit will capture the image of the visually encoded tag. Acorresponding recognition and decode application will identify theencoded data and the visual element and perform requisite decoding ofthe encoded data. In addition, the application will access device memoryto retrieve an applicable screenshot template. For example, theapplication may retrieve a template similar to the template shown inFIG. 4. The visual element(s) may then be replicated in the region ofthe template designated for such inclusion. Similar to the previouslydiscussed embodiment, it may be necessary to transform or re-format thevisual element prior to replicating the visual element in the designatedarea of the screenshot template.

FIG. 7 provides for an example of a screenshot 330 resulting fromcapturing the visually encoded data tag 300 of FIG. 6 and implementingthe screenshot template of FIG. 4, in accordance with an embodiment ofthe present invention. The first region 210 includes visual element 160.The visual element has been transformed, i.e., re-formatted to fit thesize constraints of the first region. Similar to the screenshot shown inFIG. 5, the second region 220 includes information related to thedecoded data 240.

In yet another embodiment of the present invention, the visually encodeddata tag will include designated areas for “primary” visually encodeddata and visual elements that encode user-interface display parameters.In this embodiment since both designated areas include data that isencoded, the first predefined area is specified as enclosing the“primary” encoded data or the data which the user desires to capture,decode and display. The second predefined area encloses the one orvisual elements, which, in this embodiment, encode user-interfacedisplay parameters. For example, the visual elements may be a series ofgeometric shapes with each shape related to a specific user-interfacedisplay parameter. The information may be encoded using any one ofseveral visual means, for example, shape of the visual elements,location of the visual elements, color of the elements or shading of theelements. In lieu of geometric shapes the visual elements of thisembodiment may take the form of another known symbology, such as barcodeor the like. Examples, of user-interface display parameters that may beadjusted by the coded visual elements include, but are not limited to,font type, font size, font color, background color, border type,inclusion of icons and the like. In this embodiment, the visual elementsare typically not replicated and displayed in decode data relatedscreenshot but, rather, serve to define the parameters related to thedecode data related screenshots.

FIG. 8 provides an example of a visually encoded tag 400 having adesignated area for primary visually encoded data and a designated areafor visual elements that code user-interface display parameters, inaccordance with an embodiment of the present invention. The firstpredefined area encloses visually encoded data 410, in this example,Data Matrix code, a conventional 2-dimensional barcode symbology,although code types are possible. The second predefined area enclosesone or more visual elements 420 coded with user-interface displayparameter information. In the illustrated example the visual elementsare geometric shapes and the shapes and the shading of the shapes definethe user-interface display parameters. The optional third predefinedarea encloses tag identifying data 430, in this example, the term “getmore” is included to inform a user that potentially-desirable encodedinformation is located on the tag.

FIG. 9 provides for an example of a screenshot 430 resulting fromcapturing the visually encoded data tag 400 of FIG. 8, in accordancewith an embodiment of the present invention. The template that is usedto create screenshot 430 will differ from the previously illustratedembodiments, in that, the template will only include one designated areafor enclosing the information related to the decoded data. Since thevisual elements in the tag do not represent images that will bereplicated and displayed but, rather, represent encoded user-interfacedisplay parameters, the template will not include a designated area forreplicating the visual images. The general display region 440 includesinformation related to the decoded data 450. The data encoded in thevisual elements of tag 400 is reflected in the display parameters ofgeneral display region. For example, the shading of the general displayregion indicates that the visual elements encoded a specific backgroundcolor for the user-interface display. Other parameters that may havebeen set by the encoding of the visual elements include font type, fontsize, font color and the like.

FIG. 10 provides a flow diagram of a method for capturing anddisplaying, in unison, visually encoded information and one or morevisual elements that are included on a visual tag, in accordance with anembodiment of the present invention. At step 500, a digital device thatincluding an image capture unit is directed or otherwise aimed at avisually encoded tag. The visually encoded tag includes an encoded dataportion and a visual element portion. The digital device will typicallybe a handheld digital device, such as a camera equipped cellulartelephone, a camera equipped personal digital assistant (PDA), a digitalcamera, a video camera or the like. The encoded data portion of the tagmay include any known coded symbology, such as a barcode. In oneembodiment of the invention the encoded data portion will be atwo-dimensional bar code symbology, such as PDF417, Data Matrix,MaxiCode, QR Code or the like. The visual element may take the form of abranding element, such as logo or other insignia of a product, serviceor company. Typically, the encoded data portion and the visual elementportion of the visually encoded tag will be separate, distinct regionsof the tag that will be recognizable by an associated detection anddecode algorithm. However, in one alternate embodiment, such asillustrated by FIG. 6, the encoded data portion and the visual elementportion may be integrally incorporated in a single entity.

At step 510, the digital device reads the visually encoded data. Readingthe visually encoded data entails executing a recognition applicationthat applies an algorithm to determine the location of the encoded dataportion on the tag. At step 520, the visual element portion of tag iscaptured. Capturing the visual element portion entails executing therecognition application to apply an algorithm to determine the locationof the visual element portion of the tag and, subsequently storing thevisual element in a memory unit. At optional step 530, the capturedvisual elements may be required to undergo re-formatting to prepare thevisual images for application to a screenshot or view template.Re-formatting may entail rotating or reconfiguring the visual elements,resizing the visual elements, filtering the visual elements, otherwiseadjusting visual element display parameters or the like.

The method further includes step 540, at which the digital devicedecodes the visually encoded data. The decoding of the data entailsexecuting a decoding algorithm that corresponds to the encoding methodused to encode the visual data. At step 550, once the data has beendecoded the method provides for the step of displaying, on a displayassociated with the digital device, a composite view that includes atleast a portion of the visual element portion and information related tothe decoded data. Providing for the composite view will typicallyinvolve accessing memory to retrieve a view template, extracting thevisual elements from the captured visual elements and applying thevisual elements to a predefined area of the template. In addition,information related to the decode data, such as decode confirmation orspecific decoded data, is applied to another predefined area of thetemplate.

At optional step 560, the method may include initiating networkcommunication as a result of the decoding of the visually encoded data.Initiating network communication may occur automatically based on thedecoded data or it may be initiated at the bequest of the digital deviceuser. For example, the decoded information that is displayed in a viewto the device user may suggest a form of network communication and theuser can choose to initiate the communication, defer the communicationor forego the communication. Examples of forms of network communicationinclude, but are not limited to, Short Message Service (SMS)communication, Multimedia Message Service (MMS) communication,electronic mail communication, cellular network communication, andInternet communication.

The method may further include optional step 570, storing the decodeddata and the visual element in a digital device memory. Storing thedecoded data and the visual element allows for the data to be bookmarkedand subsequently displayed at the bequest of the user. Therefore, atoptional step 580, if the method includes storing the decoded data andthe visual element. The method may also include accessing the digitaldevice memory to retrieve the stored decoded data and displaying, in asingle composite view, at least a portion of the stored visual elementand at least a portion of the stored decoded data. Additionally, storingthe visual element and the decoded data, in unison, provides for acomposite bookmark, which allows the user to browse stored data andidentify data based on the displayed visual element in the bookmark.

Thus, the present invention provides for methods, devices and computerprogram products that provide for visually encoded data to be capturedalong with one or more visual elements, decoding the captured visuallyencoded data and displaying, in one composite view, information relatedto the decoding data and the visual elements. The application thatperforms decoding of the visually encoded data may additionally providefor transforming the captured visual elements to accommodate theallocated display area of small sized display, such as the displayassociated with a handheld image capturing device. The invention is thussuited for capturing encoded data and branding elements, such as companyor product logos and displaying the branding elements in unison withdata related to the decoded information. Thus, the present inventionprovides for an efficient and user-friendly method of displaying visualelements in unison with decoded or decode confirmation information. Inaddition, the present invention provides for a one-to-one relationshipbetween the encoded data and the visual elements, such that, each tagcontaining encoded data can have individual and unique visual elementsconnected with and displayable in conjunction with the encoded data.

Many modifications and other embodiments of the invention will come tomind to one skilled in the art to which this invention pertains havingthe benefit of the teachings presented in the foregoing descriptions andthe associated drawings. Therefore, it is to be understood that theinvention is not to be limited to the specific embodiments disclosed andthat modifications and other embodiments are intended to be includedwithin the cope of the appended claims. Although specific terms areemployed herein, they are used in a generic and descriptive sense onlyand not for purposes of limitation.

1. A method comprising: directing reading, at a digital device, ofvisually encoded data included as a portion of a visual tag, the visualtag further including a visual element portion that is separate from thevisually encoded data; directing capturing, at the digital device, ofthe visual element portion; decoding, at the digital device, thevisually encoded data; and providing for displaying, on a displayassociated with the digital device, a composite view that includes atleast a portion of the visual element portion of the visual tag andinformation related to the decoded data for the visual tag.
 2. Themethod of claim 1, further comprising formatting the captured visualelement portion prior to display.
 3. The method of claim 2, whereinformatting the captured visual element portion prior to display furthercomprises performing at least one format function chosen from the groupconsisting of rotating the visual element, resizing the visual element,filtering the visual element and adjusting image parameters of thevisual element.
 4. The method of claim 1, wherein capturing the visualelement portion comprises capturing a branding element.
 5. The method ofclaim 1, wherein reading the visually encoded data comprises reading thevisually encoded data at the digital device comprising a device chosenfrom the group consisting of a cellular telephone, a personal dataassistant, a digital camera and a video camera.
 6. The method of claim1, wherein providing for displaying, on a display associated with thedigital device, comprises displaying within a single screen shot atleast a portion of the visual element and information related to thedecoded data in which the information related to the decoded datacomprises confirmation that the data has been decoded.
 7. The method ofclaim 1, wherein providing for displaying, on a display associated withthe digital device, comprises displaying within a single screen shot atleast a portion of the visual element and information related to thedecoded data in which the information related to the decoded datacomprises at least a portion of the decoded data.
 8. The method of claim1, wherein capturing the visual element portion comprises capturingbarcode data.
 9. The method of claim 1, wherein capturing the visualelement portion comprises capturing two-dimensional barcode data. 10.The method of claim 1, wherein capturing the visual element portioncomprises capturing visually encoded data chosen from the group oftwo-dimensional barcode symbologies consisting of PDF417, Datamatrix,Maxicode and QR Code.
 11. The method of claim 1, wherein capturing thevisual element portion comprises capturing the visually encoded dataportion and the visual element portion integrally incorporated in asingle entity.
 12. The method of claim 11, wherein capturing the visualelement portion comprises capturing the visually encoded data portionand the visual element portion integrally incorporated in the singleentity in which the visually encoded data portion is DataGylph encoded.13. The method of claim 1, further comprising initiating networkcommunication as a result of the decoding of the visually encoded data.14. The method of claim 13, wherein initiating network communication asa result of the decoding of the visually encoded data further definesnetwork communication as chosen from the group consisting of ShortMessage Service (SMS) communication, Multimedia Message Service (MMS)communication, electronic mail communication, cellular networkcommunication, and Internet communication.
 15. The method of claim 1,further comprising storing the decoded data and the visual element in adigital device memory.
 16. The method of claim 15, wherein storing thedecoded data and the visual element in a digital device memory furtherprovides for generating a bookmark that includes at a portion of thevisual element.
 17. The method of claim 15, further comprising accessingthe digital device memory to retrieve the stored decoded data anddisplaying on the display associated with the digital device, within asingle screen shot at least a portion of the stored visual element andat least a portion of the stored decoded data.
 18. The method of claim17, wherein accessing the digital device memory to retrieve the storeddecoded data further comprises accessing a bookmark that includes aleast a portion of the visual element and is associated with the decodeddata.
 19. A visually encoded data tag apparatus, the apparatuscomprising; a first predefined area that encloses visually encoded data;and a second predefined area adjacent to the first predefined area,wherein the second predefined area encloses one or more visual elementsincluding one or more branding elements, wherein the visually encodeddata enables, responsive to decoding thereof, displaying, on a displayassociated with a digital device, a composite view that includes atleast a portion of the one or more visual elements and informationrelated to the decoded visually encoded data.
 20. The apparatus of claim19, wherein an associated recognition application determines that thedata in the first predefined area is subject to decoding and at least aportion of the one or more visual elements is subject to user interfacedisplay.
 21. The apparatus of claim 19, wherein the second predefinedarea that encloses one or more visual elements further defines the oneor more visual elements as elements that are encoded with user-interfacedisplay parameters.
 22. The apparatus of claim 21, wherein an associatedrecognition application determines that the data in the first predefinedarea is subject to general decoding and that the elements in the secondpredefined area define user-interface display parameters that areimplemented for the display of the decoded data.
 23. The apparatus ofclaim 19, wherein the first predefined area that encloses visuallyencoded data further defines the visually encoded data as barcode data.24. The apparatus of claim 19, wherein the first predefined area thatencloses visually encoded data further defines the visually encoded dataas two-dimensional barcode data.
 25. The apparatus of claim 19, whereinthe first predefined area that encloses visually encoded data furtherdefines the visually encoded data as two-dimensional barcode data havinga barcode type chosen from the group consisting of PDF417, Data Matrix,MaxiCode and QR Code.
 26. The apparatus of claim 19, further comprisinga third predefined area adjacent to at least one of the first and secondareas, wherein the third predefined area encloses tag identifying data.27. A computer program product comprising a non transitory computerreadable medium having computer-readable program instructions embodiedin the medium, the computer-readable program instructions comprising:first instructions for determining a first predefined area of a visuallyencoded tag that encloses visually encoded data and a second predefinedarea of the visually encoded tag that encloses one or more visualelements; second instructions for decoding the visually encoded data;and third instructions for extracting at least a portion of the one ormore visual elements and applying the extracted portion to a firstpredefined area of a user-interface template, wherein the user-interfacetemplate forms the basis for a display screenshot and is selected basedon a characteristic of the visual elements, wherein the visually encodeddata enables, responsive to decoding thereof, displaying, on a displayassociated with a digital device, a composite view of that includes atleast a portion of the one or more visual elements and informationrelated to the decoded visually encoded data.
 28. The computer programproduct of claim 27, wherein the third instructions third instructionsfor extracting at least a portion of the visual elements and applyingthe extracted portion to a first predefined area of a user-interfacetemplate further provides for applying decoded data to a secondpredefined area of the user interface template.
 29. The computer programproduct of claim 27, wherein the third instructions for extracting atleast a portion of the one or more visual elements and applying theextracted portion to a first predefined area of a user-interfacetemplate further provides for transforming at least a portion of the oneor more visual elements to conform at least a portion of the one or morevisual elements to the predefined area of the user-interface template.30. The computer program product of claim 29, wherein transforming atleast a portion of the one or more visual elements is further defined asperforming a transformation operation chosen from the group consistingof rotating the visual elements, resizing the visual elements, filteringthe visual elements and adjusting image parameters of the visualelements.
 31. The computer program product of claim 30 furthercomprising fourth instructions for displaying an image that uses theuser-interface template as an outline and includes the extracted portionof the one or more visual elements in the first predefined area.
 32. Acomputer program product comprising a non-transitory computer readablemedium having computer-readable program instructions embodied in themedium, the computer-readable program instructions comprising: firstinstructions for determining a first predefined area of a visuallyencoded tag that encloses visually encoded primary data and a secondpredefined area of the visually encoded tag that encloses one or morevisual elements that encode user-interface parameter data; secondinstructions for decoding the visually encoded primary data and thevisual elements that encode user-interface parameters; and thirdinstructions for applying the decoded user-interface parameters toselect a user-interface template defining a template for display ofinformation based on the decoded visually encoded primary data, theuser-interface template being selectable based on a characteristic ofthe visual elements, wherein the visually encoded data enables,responsive to decoding thereof, displaying, on a display associated witha digital device, a composite view of that includes at least a portionof the one or more visual elements and information related to thedecoded visually encoded data.
 33. The computer program product of claim32, wherein the user-interface parameter data is chosen from the groupconsisting of font type, font size, font color, background color, anddialog template.
 34. The computer program product of claim 32, whereinthe third instructions for applying the decoded user-interfaceparameters to a user-interface template further includes instructionsfor applying information related to the decoded primary data to adesignated area of the user-interface template.
 35. The computer programproduct of claim 32, further comprising fourth instructions fordisplaying an image that uses the user-interface template as an outlineand includes the applied user-interface parameters and informationrelated to the decoded primary data.
 36. A digital device comprising: animage capture device configured to capturing an image from a visuallyencoded tag; a processing unit in data communication with the imagecapture device that receives image data from the image capture device; amemory unit in communication with the processing unit that stores one ormore user-interface templates; an application executed by the processingunit, the application non-transitory computer readable medium havingcomputer-readable program instructions embodied in the medium, thecomputer-readable program instructions comprising: first instructionsfor determining a first predefined area of the image that enclosesvisually encoded data and a second predefined area of the image thatencloses one or more visual elements; second instructions for decodingthe visually encoded data; and third instructions for extracting atleast a portion of the one or more visual elements and applying theextracted portion to a first predefined area of one of the storeduser-interface templates, the user-interface templates each defining atemplate for display of the decoded visually encoded data, theuser-interface templates being selectable based on a characteristic ofthe visual elements; and a display in communication with the processingunit that provides for the display of a screenshot based on the storeduser-interface template and including the applied portion of the one ormore visual elements.
 37. The device of claim 36, wherein the memoryunit is further defined as storing the decoded data and at least aportion of the one or more visual elements.
 38. The device of claim 36,wherein the application executed by the processing unit further includesfourth instructions for generating and storing a bookmark that includesa least a portion of the one or more visual elements and is associatedwith the decoded data.
 39. The device of claim 38, wherein the bookmarkis used by a device user to identify the decoded data when the userbrowses device memory.
 40. An apparatus comprising: a processor andmemory including computer program code, the memory and the computerprogram code configured to, with the processor, cause the apparatus to:direct reading of visually encoded data included as a portion of avisual tag, the visual tag further including a visual element portion;direct capture of the visual element portion; decode the visuallyencoded data; and provide for displaying, on a display of a digitaldevice performing the reading, capturing and decoding, a composite viewthat includes at least a portion of the visual element portion of thevisual tag and information related to the decoded data for the visualtag.
 41. The apparatus of claim 40, wherein the memory and computerprogram code are further configured to, with the processor, cause theapparatus to format the captured visual element portion prior todisplay.
 42. The apparatus of claim 40, wherein the memory and computerprogram code being configured to, with the processor, cause theapparatus to provide for displaying includes causing the apparatus todisplay within a single screen shot at least a portion of the visualelement and information related to the decoded data in which theinformation related to the decoded data comprises at least a portion ofthe decoded data.